Web Survey Bibliography
Relevance & Research Question
The market research industry is wedded to quota controls. We apply Age and Gender quotas without a second thought as to why or indeed whether they are doing any good at all. Our argument is that, in the modern online sampling world, a different set of stratifications must be applied and our old assumptions simply do not apply. Why not? The answer, in common with so many of the problems in sampling in online research, lies in the frame. The frame in traditional research was close to the population; therefore a quota controlled random sample would tend to produce samples that, within the quota strata, also contained representative numbers of all other attitudes and behaviours. This is not the case with online access panels.
Methods and Data
Our experiment uses our US panel; the topic, eye colour, is unrelated to Age and Gender but is strongly related to Ethnicity. We have conducted 2 samples. The first strictly controlled on Age, Gender and Region, the second controlled on Ethnicity alone. Our Age Gender Region ‘nat rep’ sample should underestimate the number with brown eyes. The Ethnicity we expect to estimate eye colour extremely well. At the same time a third sample will be drawn which is simply “random enough”. Our expectation is that this sample will also under-perform on eye colour but will equal the findings from “nat rep” sample 1. A second experiment will be undertaken where the variable of interest is unrelated to anything –left- or right-handedness. Our hypothesis is that all three samples will perform equally well.
Results
The results are precisely as predicted.
Added Value
Researchers, particularly in the commercial world, apply quota controls to ensure “representivity” as a matter of practice, they do it because they have been told to, it is part of the folklore of market research. This is not sustainable in a world where we are no longer dealing with essentially incomplete frames. More science and less folklore needs to be applied to make the best of an increasing unscientific world.
Conference Homepage (abstract) / (presentation)
Web survey bibliography - 2011 (358)
- The Validity of Surveys: Online and Offline; 2016; Wiersma, W.
- Computer science security research and human subjects: Emerging considerations for research ethics boards...; 2013; Buchanan, E. A., Aycock, J., Dexter, S., Dittrich, D., Hvizdak, E. E.
- Multiple Sources of Nonobservation Error in Telephone Surveys: Coverage and Nonresponse; 2011; Peytchev, A.; Carley-Baxter, L. R.; Black, M. C.
- Online Questionnaires for Outbreak Investigations; 2011; Parry, A. E.; Johnson, D. R.; Byron-Gray, K.; Raupach, J. C. A.; McPherson, M.
- Inventory of published research: Response burden measurement and reduction in official business statistics...; 2011; Giesen, D. & Snijkers, G. (Eds.), Bavdaz, M., Bergstrom, Y., Gravem, D. F., Haraldsen, G., Hedlin, D...
- Effects of speeding on satisficing in Mixed-Mode Surveys; 2011; Bathelt, S., Bauknecht, J.
- Using Research-Based Practices to Increase Response Rates of Web-Based Surveys; 2011; Perkins, R. A.
- Using break-offs in web interviews for predicting web response in mixed mode surveys; 2011; Beukenhorst, D.
- Web panels in Slovenia; 2011; Lenar, J.
- Traditional and non-traditional treatments for autism spectrum disorder with seizures: an on-line survey...; 2011; Frye, R. E., Sreenivasula, S., Adams, J. B.
- Understanding the new digital divide—A typology of Internet users in Europe; 2011; Brandtzæg, P.B.; Heim, J.; Karahasanoviæ, A.
- Patients’ attitudes toward side effects of antidepressants: an Internet survey; 2011; Kikuchi, T., Uchida, H., Suzuki, T., Watanabe, K., Kashima, H.
- Web-based or paper-based surveys: a quandary?; 2011; Bennett, L., Sid Nair, C.
- Refining the Total Survey Error Perspective; 2011; Smith, T. W.
- ELIPSS: Étude Longitudinale par Internet Pour les Sciences Sociales; 2011; Legleye, S., Lesnard, L.
- Less questions, more data: Revitalizing the european currency in single source affluent audience measurement...; 2011; Hartman, H.
- Linking website exposure data to survey data: A single-source solution; 2011; Krahn, J., Landi, J., Melton, E.
- Inference in surveys with sequential mixed-mode data collection; 2011; Buelens, B., van der Brakel, J.
- Using a Probability-based Online Panel to Survey American Jews; 2011; Wright, G., Phillips, B. T., Tobias, J., Peugh, J., Semans, K.
- Choice of Content Presentation Mode in Web-Based Survey Administration; 2011; Osborn, L., Mansfield, W., Ramirez, C. M., Lacey, J. N., etc.
- Seasonal Yield Variation and Related Response Patterns in Address-based Mail Samples; 2011; DiSogra, C., Hendarwan, E.
- Gender-specific on-line shopping preferences; 2011; Ulbrich, F., Christensen, T., Stankus, L.
- Mixing modes in the LFS - Computer-assisted, cost effective and respondent friendly; 2011; Koerner, T., van der Valk, J.
- Peanuts and Monkeys: Incentivisation and engagement in online access panels; 2011; Marks, B.
- Establishing Cross-National Equivalence of Measures of Xenophobia: Evidence from Probing in Web Surveys...; 2011; Braun, M., Behr, D., Kaczmirek, L.
- Methodological challenges in the use of the Internet for scientific research: Ten solutions and recommendations...; 2011; Reips, U.-D., Buchanan, T., Krantz, J. H., McGrawn, K.Reips, U.-D.
- Search and email still top the list of most popular online activities; 2011; Purcell, K.
- Using Internet in Stated Preference Surveys: A Review and Comparison of Survey Modes; 2011; Lindhjem, H., Navrud, S.
- On the experience and evidence about mixing modes of data collection in large-scale surveys where the...; 2011; Dex, S., Gumy, J.
- Survey Gamification: Old Wine in New Bottles?; 2011; Baker, R. P.
- The Game Experiments: Researching how gaming techniques can be used to improve the quality of feedback...; 2011; Sleep, D., Puleston, J.
- Statistical Estimation of Word Acquisition With Application to Readability Prediction; 2011; Kidwell, P., Lebanon, G., Collins-Thompson, K.
- What is Probit; 2011
- Voice-of-the-customer marketing: A revolutionary 5-step process to create customers who care, spend,...; 2011; Roman, E.
- User agent; 2011
- Unpublisihed internal Google report on break off rates by device type; 2011; Callegaro, M.
- Toward wiser public judgment; 2011; Yankelovich, D., Friedman, W.
- The impact of cookie deletion on site-server and ad-server metrics in Australia. An empirical comScore...; 2011
- The changing role of address-based sampling in survey research; 2011; Iannacchione, V. G.
- State of mobile measurement; 2011; Gluck, M.
- Some issues in the application of latent class models for questionnaire design; 2011; Biemer, P. P., Berzofsky, M.
- Self-administered mobile surveys; 2011; Bosnjak, M.
- SDSC Announces scalable, high-performance data storage cloud; 2011
- Ratings and audience measurement; 2011; Napoli, P. M.
- Randomized response models in survey sampling. Randomized response models; 2011; Hussain, Z.
- Online survey research: Findings, best practices, and future research. Report prepared for the Advertising...; 2011; Vannette, D.
- Online survey research: Findings, Best practices, and future research; 2011
- New Esomar survey on use of cookies and tracking technologies; 2011
- Mobile, webmail, desktops: Where are we viewing email now?; 2011
- Measuring americans' issue priorities. A new version of the most important problem question reveals...; 2011; Yeager, D. S., Larson, S. B., Krosnick, J. A., Tompson, T.